Exynos Backend for Executorch to bring up on Exynos SoC #13677
Conversation
Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13677
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New FailureAs of commit 4994e6b with merge base 6c12956 ( NEW FAILURE - The following job has failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
bot
added
the
CLA Signed
|
@pytorchbot label "release notes: Exynos" |
|
Didn't find following labels among repository labels: release notes: Exynos,partner:Samsung |
|
@pytorchbot label "release notes: Exynos" |
|
Didn't find following labels among repository labels: release notes: Exynos |
- Provides a Python interface for compiling NN models via LiteCore in ahead-of-time compilation. Co-authored-by: chong-chen <chong.chen@samsung.com> Signed-off-by: chong-chen <chong.chen@samsung.com> Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
Add Op wrapper and tensor wrapper for Converter of LiteCore. Also, include the op param wrapper and quantize param wrapper. Co-authored-by: chong-chen <chong.chen@samsung.com> Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
- For Exynos AI LiteCore, EXIR will be serialized and converted. Signed-off-by: chong-chen <chong.chen@samsung.com> Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
- It can be converted op or graph Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
Implement backend and executor for exynos. Co-authored-by: chong-chen <chong.chen@samsung.com> Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
- Add the enn executor and backends Signed-off-by: chong-chen <chong.chen@samsung.com> Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
- it is descripted how to compile/use Exynos backend. Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
Jiseong-oh
force-pushed
the
exynos-backend
branch
from
e98768d to
66bd9ea
Compare
- update class name Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
- This script can be supported for building exynos backend easily Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
- Add exynos backend on executorch Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
mergennachin
requested review from
GregoryComer,
SS-JIA,
digantdesai,
kimishpatel and
mergennachin
| fout.close(); | ||
| } | ||
|
|
||
| int main(int argc, char** argv) { |
There was a problem hiding this comment.
If you dont feel like you need to be able to have your runner execute on baremetal you could use extension/Module rather then dealing with the lower level ET apis like dataloader, method, methodmeta etc
There was a problem hiding this comment.
Thanks @JacobSzwejbka for reviewing.
However, do you have any examples which uses high level APIs (extension) in runner? I briefly reviewed the source code of runner in XNNPACK (portable) or QNN for reference, they seem to use similar APIs to each other from my point of view. If you link any references for that, it might be very helpful to implement.
We are preparing the next commits such as supporting dummy-inputs (using extension) if user doesn't indicate input test file.
There was a problem hiding this comment.
Ah Im not sure we do have any runner's written against these apis. xnnpack / portable's runner predates their existence and QNN's seems copy pasted from that one with some QNN specific changes.
In general they should be pretty easy to use though I think. Just need this constructor https://github.com/pytorch/executorch/blob/main/extension/module/module.h#L63
and then call
https://github.com/pytorch/executorch/blob/main/extension/module/module.h#L379
extension/tensor is just to dramatically simplify creating ET tensors. Its an owning wrapper that manages the buffers that back the metadata like sizes and strides. from_blob is probably the api you would want to call from there.
There was a problem hiding this comment.
@JacobSzwejbka
We'll try to apply your comment for the next PR. thanks!
mergennachin
added
partner: samsung
- this option will be revmoed after stable link is defined. Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
| NDK_INSTALLATION_DIR=/opt/ndk | ||
| rm -rf "${NDK_INSTALLATION_DIR}" && sudo mkdir -p "${NDK_INSTALLATION_DIR}" | ||
| ANDROID_NDK_VERSION=r27b | ||
|
|
||
| pushd . | ||
| cd /tmp | ||
| curl -Os --retry 3 "https://ossci-android.s3.amazonaws.com/android-ndk-${ANDROID_NDK_VERSION}-linux.zip" | ||
| unzip -qo "android-ndk-${ANDROID_NDK_VERSION}-linux.zip" | ||
|
|
||
| # Print the content for manual verification | ||
| ls -lah "android-ndk-${ANDROID_NDK_VERSION}" | ||
| sudo mv "android-ndk-${ANDROID_NDK_VERSION}"/* "${NDK_INSTALLATION_DIR}" | ||
| popd | ||
| # build Exynos backend | ||
| export ANDROID_NDK_ROOT=${ANDROID_NDK_ROOT:-/opt/ndk} |
There was a problem hiding this comment.
We have another docker image that already contains android ndk so that you don't have do download and install.
You'll have faster CI turnaround time. Here's how the docker image is configured
executorch/.ci/docker/build.sh
Lines 59 to 64 in cec1400
executorch/.ci/docker/ubuntu/Dockerfile
Lines 79 to 83 in cec1400
There was a problem hiding this comment.
Could i try to change the docker next pr? we can test to build with our SDK on this docker after this pr is merged.
Actually, we are preparing to change to executorch-ubuntu-22.04-clang12-android.
There was a problem hiding this comment.
Could i try to change the docker next pr?
Okay, please create an issue into the board so that we can track ExecuTorch Samsung (view)
There was a problem hiding this comment.
I created a project board to track all issues related to
https://github.com/orgs/pytorch/projects/140/views/3
Each issue contains a milestone (e.g., 1.0.0, 1.1.0)
Majority of them can be done for 1.1.0 after this PR lands. The only one that I want to land before 9/12 branch cut is #13999
| add_library(enn_logging STATIC) | ||
|
|
||
|
|
||
| if(${CMAKE_SYSTEM_PROCESSOR} MATCHES "x86_64") |
There was a problem hiding this comment.
I created an issue to track here as a follow-up PR after this one lands: #14000
| - Exynos 2500 (E9955) | ||
|
|
||
| ### Supported Inference Type | ||
| - Quantized (i8/u8/i16/u16) |
There was a problem hiding this comment.
Created an issue to track: #14003
| ### Build AOT Targets | ||
| Generated python artifacts allow user call `Compile` interface to lower a model to ENN backend in python script. | ||
| ```bash | ||
| ./backends/samsung/build.sh -b x86_64 |
There was a problem hiding this comment.
For e2e integration test, created an issue to track: #14002
| ```bash | ||
| cmake extension/android \ | ||
| -DCMAKE_TOOLCHAIN_FILE="${ANDROID_NDK}/build/cmake/android.toolchain.cmake" \ | ||
| -DANDROID_ABI="${ANDROID_ABI}" \ | ||
| -DCMAKE_INSTALL_PREFIX=cmake-android-out \ | ||
| -Bcmake-android-out/extension/android |
There was a problem hiding this comment.
cc @Jiseong-oh please take a look at Kimish's question
| is deploying torch model run with exynos NPU/DSP. | ||
|
|
||
| This backend is implemented on the top of [EXYNOS_LITECORE](https://soc-developer.semiconductor.samsung.com/global/development/light-core) | ||
| Please prepare the SDK before you start, it is important to code compilation and runtime. |
There was a problem hiding this comment.
Created an issue to track: #14004
Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
Signed-off-by: jiseong.oh <jiseong.oh@samsung.com>
Could't I have a chance to merge next pr before 9/12 branch cut? Acutally, I am preparing to next pr for supporting other float models and ops now. |
Yeah, let's try that. If it's possible to land before the branch cut, that's fine too since we have the basic integration. . Please create an issue and add to this board ExecuTorch Samsung (view) |
There was a problem hiding this comment.
@Jiseong-oh --
- Thanks for the CI fix, we will try to land this PR now.
- Also, you have full access to the project board (https://github.com/orgs/pytorch/projects/140/views/3). You can create new issues and add to the board as you fit. I'd encourage you to proactively use the board to communicate issues and plans, such as what needs to be done as follow-up PR before branch cut, what can be done afterwards etc.
- please try to reproduce. you probably don't have exynos device, but at least reproduce the AOT part.
- please import internally and double check it doesn't break any internal test case. Probably it won't since it is an isolated change, but double checking would be good.
cc @kimishpatel , @digantdesai , @JacobSzwejbka @GregoryComer -- any thoughts? if you have major feedback perhaps you can just create an issue in this board (https://github.com/orgs/pytorch/projects/140/views/3), with appropriate milestone and priority attached and they can do as follow-up PRs, if necessary.
|
I checked out the PR and was able to build all the components. I was able to build the ENN executor runner and at least run it with However, I was not able to reproduce exporting models with the It seems that there is a dependency on glog/gflags. To attempt to workaround it, I tested building glog/gflags from source (I initially tried to install via Using I raised this with @Jiseong-oh, and it seems that the I'll merge this PR for now so that Samsung folks can begin working on the next PR. Getting access to a Ubuntu machine to properly reproduce may not be easy, please let me know if anyone has any ideas. |
9 participants
Summary
This PR introduces support for the Exynos backend of Executorch, can be inferenced on Exynos acceletors.
Exynos Backend is a crucial component designed to optimize the performance of AI models on Exynos chipsets. Leveraging Exynos AI LiteCore, it enables comprehensive support for NPU/DSP, CPU, and GPU, ensuring seamless execution of AI workloads on Exynos-based devices.
Below is a description of the features:
Exynos Backend : The backends/samsung directory were made by ENN Partitioner, I/F for communication with Exynos LiteCore, runtime backend files and device-specific optimization. See backend/samsung/README.md for usage
Exynos examples: The examples/samsung directory privides scripts for AOT compilation, quantization and so on. please refer examples/samsung/README.md
Test plan
This PR is tested with Exynos Backend on NPU/DSP/CPU/GPU of Exynos 2500 SoC. To run the layer and models tests with README.md
cc: @mergennachin @SS-JIA @digantdesai @kimishpatel